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EDUCATION POSITION OR POLICY 



Introduction 



\ i f One occasionally comes across papers sup^gesting various kinds of 

nonlinear transformations of observed data in an experiment, as a means of 
reducing or eliminating effects attributable to interaction among the factors 
of the cxperiir.ental design; such procedures seem especially to be advocated 
where the interactions found in the original data are difficult to interpret. 

Q'^^ While there may sonretimes be good reasons for examining transformations of the 
, .J data (reasons related, e.g., to deep questions about the nature and moaning of 



the measured variable), it is probably unwise to attempt to eliminate inter- 
actions solely because of interpretative difficulty: the data, after all, may 
be trying to tell the investigator something,' An alternative approach is 
presented here: Because the compactness and efficiency of complete balanced 



designs arise frora the use of every available cell mean in the estimation of 



every effect implied by the formal design, one cell which is systematically 
atypical (or "strange") v;ith respect to the other cells can contaminate all 
reported sources of variation in the usual analysis of variance (ANOVA) summary 
table. It follows that if that cell can be identified, and its mean ''adjusted" 
for the suspected "strangeness," the summary table can be similarly adjusted 
and the resultant pattern of significant effects compared with the originally 
obtained pattern. If the new results are more consistent with theoretical 
arguments, and if the strangeness implied in the "adjusted" cell also makes 
some theoretic sense, then an hypothesis asserting the existence of "strange- 
ness," or of a synergistic effect, in that ceil is tenable. 

« 1 
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(Since in general that particular hypothesis will be only one among many 
alternative hypotheses, some of which may be equally attractive, the proper 
outcome in most cases will be a new experiment desif>ned specially to distinguish 
among these competitors. Only rarely will an unequivocal explanation be clearly 
implies by the data; and, usually, such explanations will be evident without 
a statistical analysis.) 

Synergistic effects 

For the purposes of this discussion, a synergy , or synergistic inter- 
action, is taken to mean a unique effect in a single cell of an MOVA design, 
which acts to change the value of the population mean in that cell (but not in 
others), and docs not affect the w^ithin-cell variation about the cell mean. 
In effect, postulating a synergy in a design amounts to adding another term to 
the AIJOVA model while deleting one of the usual terms of the model (in a 2 
design) or removing one degree of freedom from one of the usual terms (when 
there are factors with more than two levels) . 

By way of illustration, consider an electrical circuit containing a 
battery, a light bulb, and three switches, all connected in series. Each switch 
has two positions (labelled, perhaps, "high" and "low") and corresponds to a 
two-level design factor. Only when all three switches are in their respective 
"on" positions (which may or may not correspond to the label "high") does elec- 
tric current flow through the circuit and cause the bulb to glow. (The depen- 
dent variable corresponds either to the illumination provided by the bulb, or 
to the amount of current flowing in the circuit). For any other settings of 
the switches — i.e., if any one or more of them are "off" — no current will 
flow and the bulb will remain unlit. Again, in a circuit containing a battery 
and a bulb in series with a set of three switches, but where the switches are 
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connected in parallel, current will flow and the bulb will light up unless all 
three switches are *'off." 

Oncatalysis 

Neale and Licbcrt (1973, page 63), in what appears to be an egregious 
misunderstanding of the nature of catalysis in chemistry, use the term "cata- 
lytic interaction" to describe the situation wherein "two or more treatments 
are effective only when they occur together." The paradigm is an experimental 
outcome in which the value of the dependent variable is constant over all cells 
of a design, with one exception; in the exceptional cell the dependent variable 
takes on a substantially different value. Since in such a paradigm there is 
no obvious role for a "catalyst," in the sense of a substance (or an analogue 
of a substance) whose presence is necessary fcr the reaction (interaction) to 
occur but wliich is itself unchanged by the reaction, the term synergistic in-- 
teraction (or, more simply, synergy) is preferred herein to describe the case 
of a dependent variable whose value changes sharply as an apparent result cf 
the coincidental occurrence of particular values of each independent variable. 
Wliile the notion is clearly extensible to synergies in a number of cells in a 
design, this paper is addressed only to detecting and interpreting a synergy 
in one cell; the simultaneous existence of real main effects and of some non- 
synergistic interactions is permitted, however. 

An arbitrary (hypothetical) example 

Imagine a 2^ MOVA design in which there arc no differences in cell 
means for seven of the eight cells, but the eighth cell is remarkably (in the 
sense of Saunders (1970)) different from the others. Uecause all the sources 
of variation reported in the ANOVA are equivalent to single contrasts comparing 
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four of the cells to the other four cells, the exceptional cell (and it alone) 
will contribute to all seven sources of variation reportetl in the standard 
ANOVA: A, li, C, AB, AC, BC, anc ABC. If any effect is siv.niHcant, all arc 
significant. Such a situation is depleted in Figure 1(a), v/ith artificial 
"data*' reported in Table 1(a). In this case, it is clearly more parsimonious, 
and probably more informative for the developi lent of theory, to describe the 
observed pattern of results as due to a sinple synerpy anonj.- tht three design 
variables, rather than as reflectinp seven independent sources of variation. 
(Of course, it is impossible purely on the basis of the data to distinp,uish 
between the two explanations; both '^explain" the results equally quantitatively. 
One nay prefer one explanation or another on the basis of parsimony, theory, or 
sheer intuition, but a specially designed experiment would be required to make 
an empirical (or statistical) distinction.) 

Now suppose the situation is the same as just described, but there is 
also a 'true* main effect due to factor B, in the opposite direction to the 
'spurious' effect caused by the synergy in cell (122). Then the variation 
attributed to B in the ANOVA is reduced (in the present case, eliminated al- 
together because of using a 'true' effect equal in size to the 'spurious' ef- 
fect), but the other six source^ remain 'significant' in the ANOVA. This case 
is shown in Table 1(b) and Figure 1(b). The addition of other 'true' effects 
to the design will either increase or diminish the reported strength of the 
effects, depending on whether the 'true' effects are in the same direction as 
the 'spurious' effect generated by the synergy, or in the opposite direction. 
As more 'true' effects are added, the 'spurious' effect of the synergy is more 
and more concealed by the real effects; eventually it becomes more parsimonious 
(but not necessarily truer) to Interpret the data in terms of the ordinary ANOVA 
results than in terms of a synergy plus some 'real' effects. See Table 1, (c) 
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to (h), and Figure i (c) to (h) . [The dashed lines in Figure 1 represent the 
patterns that would have been observed had no synergy been present. ] 

By the time we get to cases (d) and (e) in Table 1» it is difficult 
to discern a ^lear synergy in the tables , although the effect is somewhat more 
apparent in Figure 1. Among the strategies that might be used to make the 
strangeness of cell (122) more visible are (1) representing the values in each 
cell as deviations from the smallest cell mean and (11) representing them as 
deviations from the grand mean of the observed values. Strategy (i) Is shown 
in Table 1, (a') to (h'); strategy (ii) by Table 1, (a") to (h"). Up to about 
cases (d) and (e) the deviations suggest a 'strangeness' about cell (122), but 
the suggestiveness decreases as more 'real' effects are added. 

Detection of synergies 

From the definition given above (see page 2) , the detection of a syn^ 

ergy is equivalent to the estimation of the change in the value of a cell mean 

as a result of the proposed synergy. The estimation requires one degree of 

freedom, which must be found among the degrees of freedom associated with treat- 
k 

ments. In 2 designs, detection of a synergy therefore requires assuming that 
one of the usual sources of variation does not exist. Since an assumption of 
this kind is often implied by underlying theoretical arguments (or made tacitly 
by the investigator) with respect to higher-order interactions, the requirement 
Is unlikely to be prohibitive, especially for k > 3. Where design variables 
have more than two levels, the required degree of freedom may be obtained by 
imposing some constraint (s) on one or more sources of variation^ rather than 
requiring one source to disappear altogether. 

For 2 designs the following procedure is simple and straightforward: 
first, select a source of variation which "ought" not to exist (the selection 
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may be based on various criteria: a theory that implies absence of a particular 
interaction (or main effect, for that matter), or starting with the smallea re- 
ported mean square, for example). In a 2 design, this source will have one 
degree of freedom. Write the formal contrast representing? this source, set it 
to zero, and solve for the mean of a selected cell la terms of the other cell 
means. (As before, selection of the cell may be based on various criteria.) 
The effect of this procedure is to attribute all variation apparently due to a 
particular source — a line in the usual ANOVA table — to a synergy, and to 
use the degree of freedom usually associated with that source to estimate the 
size of the synergy. Finally, recalculate the ANOVA, using the newly calculated 
cell mean instead of the original value in the cell selected for adjustment, 
and compare the pattern of results with that obtained from the raw data. If 
the new results are more satisfying in terms of theoretical considerations, and 
a synergy in the particular cell selected makes some kind of sense, and if the 
new results are more parsimonious than the original (in the sense either of re* 
quiring fewer sources to explain the variation observed, or of producing & 
smaller value for the total treatment sum of squares (SS^^), then a synergy in 
the selected cell is a tenable hypothesis. 

There may, however, be other tenable hypotheses meeting these criteria; 
it is therefore to be recommended that the procedure just described with respect 
to one selected cell in the design be repeated with each of the other cells in 
turn. If more than one tenable hypothesis results (which will nearly always be 
the case, since the usual ANOVA sources of variation presumably represent an 
a priori set of tenable hypotheses), the task is then to design one or more ex- 
periments which will be capable of distinguishing among the several hypotheses. 

The procedures just described may most easily be understood by con- 
sidering their application to some real data. 
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An empirical example 

In an experiment investigating the degree to which individuals die-* 
play understanding of logical principles and their application to a certain 
kind of inferential problem, Bracewell and Hidi (1973) observed that in one of 
their eight (« 2^) conditions nearly perfect performance routinely occurred, 
while the other seven conditions displayed substantially poorer performance 
(with systematic differences among them). The ANOVA sumoary indicated signif- 
icant main effects due to factors A and C of the three-way design, and no 
effect due to B, all more or less as expected; but in addition a significant 
EC interaction was reported which was unexpected and difficult to interpret* 
There was, however, a reasonable basis for interpreting the superior performance 
in the one cell mentioned above; under the suspicion that performance in this 
cell might be sufficiently unlike that in the other cells to generate a spurious 

interaction, an "adjusted** mean for that cell was calculated, on the basis of 

[21 

denying the presence of a three-way interaction. The original data are re- 
ported in the second line of Table 2, labelled '*Raw Mean," and are displayed 
graphically in Figure 2. The original sum of squares (SS) for each source of 
variation appears in the rightmost column of Table 2, in lines 1-7; their sum 
appears in line 8 as S^^^* 

Using the "adjusted" value for cell (111), the ANOVA was repeated. 
The adjusted mean is indicated in the lower part of Table 2, and the contrast 
values representing each source of variation appear in the column beneath the 
adjusted mean. Again the main effects A and C were significant (both less 
strongly than before), but there were no other signiflcan effects. All mean 
squares were smaller than they were originally; those for effects i>, AB, AC, 
and BC ranged from 0 to 31% of their original values; expressed in terms of 
the error mean square ^MS^) the absolute decrements In mean squares for these 

o 
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four effects ranged from 1.36 MS^ to 4.78 MS-,. Also, SS-, was considerably 
reduced, from 275.3 to 111.3, about 40% of its original value. The postulated 
synergy, therefore, could be said to account for 60% of the total ^^j^t with 
a SS of 164.0, considerably larger than any other effect; even though the 
actual difference in the cell mean was only b, the value of the original ABC 
.contrast- 

At first glance, these results seemed impressive evidence for the 

jexistence of a unique effect in cell (111), which might be explained in terms 

i 

of a synergy representing the action of the three factors in the design. To 

i 

\ 

iee whether such a synergy operating in some other cell of the design might be 
in equally attractive explanation of the behaviour of the data, the same pro- 
Cedure was applied to every other cell. The adjusted means and adjusted con- 
trast values (in lieu of the several SS's) appear in the lower part of Table 2, 
together with the value of SS^^ on line 8. Each set of means, with one cell 
mean adjusted, is displayed in Figure 2. Besides allowing comparisons among 
all possible unique effects as possible explanatory devices, the procedure also 
permits examining the pattern of siginif leant effects, to see (e.g.) whether 
the main effects A and C always turned up, or whether one or the other of 
them also disappeared under some hypothesized synergies* As Table 2 shows, 
these two main effects always showed up, with A usually stronger than C; 
and in all cases except adjusting cell (111) , at least one other effect appeared 
(and usually at least one of these additional effects were significant at the 
.01 level). On the basis of parsimony, then, one would prefer the original 
hypothesis that a unique effect or synergy was operating in cell (111), together 
with two easily interpretable main effects, and no interactions other than the 
synergy; since any other synergy requires one or more interactions to exist* 



- 9 - 



Another possible criterion Is the total SS^^ resulting from adjustinent 
of a cell mean* Wlille It Is possible to Imagine a unique effect operating In 
such a way as to mask or suppress real general effects (main or Interactions) 
due to the design factors, and therefore to reduce the total ^^j^ reported 
In the original ANOVA table from what that total *'ought*' to be, one would need 
compelling substantive or logical reasons to accept the existence of such a 
^'suppressor Ordinarily, one would expect a parsimonious explanation to re- 
duce the adj us ted ^S^j. to a lower value than the original value* On examining 
Table 2, we see that In all cases except cells (111) and (222) the SS^^ after 

adjustment Is larger than before — by amounts ranging from 6*14 MS^ to 26.60 

E 

MS^« Since In this case the compelling reasons supported cell (111) rather 
than any others, and since adjusting cell (111) both reduces SS^^ the most 
and results In the smallest number of significant effects, the most satisfying 
explanation of the original data adduces a synergy or unique effect In cell (ill) 
plus main effects due to factors A and 

Extension to several contrasts 

In the case of designs Involving factors with more than two levels, 

the usual ANOVA sources of variation will have more than one degree of freedom* 

k 

For such cases the same kind of procadure and reasoning apply as for 2 designs; 
but having selected a source of variation which "ought'' not to exist (or which 
the Investigator wants to minimize) and which has / degrees of freedom, one 
may either write the total SS for that source as an explicit function of the 
cell means, and then minimize the SS with respect to each cell mean In turn; 
or one may write the SS as a sum of D?, i » 1,***» / (see, e.g., Guenther, 
1954, §2-12) corresponding to / orthogonal contrasts, set one of those 
contrasts to zero, and solve for the cell mean(s) to be adjusted. In either 
case there remain /-I degrees of freedom for variation due to that source 
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above and beyond the effects of the hypothesized synergy. As for the 2 designs » 
results of adjusting for hypothetical synergies would be evaluated In terms of 
parsimony of slgnf leant effects, parsimony of total ^^^^p > Implications for 
theory of synergies in particular cells, and general Interpretablllty ; and 
since this is a variety of poet hoc analysis, such evaluations ought in general 
to lead to refined experimental designs and additional hypotheses to be tested 
rather than to firm conclusions. 

The principle of minimizing a SS having / degrees of freedom with 
respect to a cell c:ean logically implies the possibility of minimizing a SS 
composed of several sources of variation with respect to a cell mean. While 
no obvious examples spring immediately to mind where a procedure of this kind 
would be appropriate, such examples may exists 

Rules of thumb 

The procedures described in this paper for detecting synergistic ef- 
fects in analysis of variance designs are summarized in the rules of thumb dis- 
played in Table 3. 

Implications 

Ordinarily, the impact of substantive theory on an experimental study 
is largely limited to the design of the experiment: the number and selection 
of the design variables, the number and selection o£ discrete levels (values) 
of each variable, and perhaps specification of the effect size (see Cohen» 1973) 
against which reasonable power is desired. After the experiment has been car- 
ried out and the data have been analyzed, the investigator typically attempts 
to explain the results of the ANOVA in the light of theory, or (g)he may attempt 
to illuminate (or modify) theory on the basis of the experimental results. The 
analysis of data, however, is typically carried out in a purely formal (not to 
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say mechanical) way, without reference to the substantive area of the investi- 
gation nor, often, to theory which may imply certain expectations about the 
results* 

The techniques discussed in this paper make it possible to apply 
some kinds of theoretical insights to the data-analysis phase of a study: 
either by seeking synergistic effects implied or predicted by theory, or by 
seeking evidence of synergies as alternative explanations for results which 
would otherwise imply the existence of interactions where theory would deny 
such interactions. In complex factorial designs particularly, where high* 
order Interactions are often called "uninterpretable," these techniques may 
often permit more appealing explanations and interpretations of the experimental 
results* 
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NOTES 

[1] Neale and Liebert (1973) also defis:c "terminativc interactions," 

v;herein "two or more variables arc clearly effective in nodifyinp. 
behavior, but,^ when combined, tiicir effect is not increased over what either 
alone could do/* Analytically, this is not different fro;r. n "cat.-ilytic 
interactioii*' : the apparent difference seems to depend wholly on wliether one 
starts by considering pairwise cell contrasts wiiich arc substantially (or 
"clearly"?) different from zero, or by considerinj contrasts wliich are not 
different from zero; and possibly on whether the non-xero contrasts are 
"positive'' or "negative." Such distinctions appear unreasonably arbitrary. 

[2] Notice that the original, or "raw," three-way interaction was one 

of the two smallest effects in the analysis (the other being the AB 
interaction). Observing that the original AIU: contrast had the value 8, 
we see that the "adjustment" consists in subtractinp 8 fron cell (111), since 
the contrast coefficient for that cell in the ABC contrast was +1. Similarly, 
the adjusted mean for each other cell is either 8 less or 8 more than its "raw" 
value, depending on whether its coefficient in the A13C contrast was +1 or -1, 
Incidentally, another check on the reasonableness of an hypothesis of synergy 
is the adusted mean in a given cell; if it is "adjusted" out of the possible 
range of observable values, the hypothesis is a little hard to take! Since in 
the present example the minimum possible cell mean was 12, three of the "synergies" 
examined are in this sense nonsensical. 
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FIGURE 1. An Arbitrary Example. Data from Table 1. 

[Dotted lines indicate relationships that would occur in the absence 
of the synergy in cell (122).] 
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